10 things you need to know about Date and Time in Python with datetime, pytz, dateutil & timedelta
Dates and Time probably sounds like an easy concept, until you have to deal with users and data from around the world.
Then you realize, it’s actually complicated!
TL;DR: the cheatsheet
UPDATE:
Since Python 3.9, ZoneInfo is available and it solves many of the issues seen with pytz, so DON’T USE pytz, use ZoneInfo
1. Parse date strings
import datetime
import dateutil.parserformat = '%Y-%m-%dT%H:%M:%S%z'
datestring = '2016-09-20T16:43:45-07:00'
# d = dateutil.parser.parse(datestring) # python 2.7
d = datetime.datetime.strptime(datestring, format) #3.2+
2. ISO-8601 date string to UTC datetime object
import datetime
import dateutil.parser
from zoneinfo import ZoneInfo
utc = ZoneInfo('UTC')
format = '%Y-%m-%dT%H:%M:%S%z'
datestring = '2016-09-20T16:43:45-07:00'
d = datetime.datetime.strptime(datestring, format) #3.2+
d = d.replace(tzinfo=utc) - d.utcoffset()
>>> datetime.datetime(2016, 9, 20, 23, 43, 45, tzinfo=<UTC>)3. UTC Timestamp (now)
import datetimenow = datetime.datetime.now()
now.timestamp()
>>> 1505329554.617216
# now is a naive datetime. .timestamp() will convert to UTC unix timestamp considering the system local timezone
4. UTC Timestamp from naive date string
import datetime# naive datetime
d = datetime.datetime.strptime('01/12/2011', '%d/%m/%Y')
>>> datetime.datetime(2011, 12, 1, 0, 0)
# Note this is in the local timezone
d.timestamp()
>>> 1322726400.0# when the date is not in the proper timezone, if needs to be converted
from zoneinfo import ZoneInfo# add proper timezone for the date
pst = ZoneInfo('America/Los_Angeles')
d = d.replace(tzinfo=pst)
>>> datetime.datetime(2011, 12, 1, 0, 0, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)d.timestamp()
>>> 1322726400.0
5. Add timezone to a naive datetime
import datetime
from zoneinfo import ZoneInfo# naive datetime
d = datetime.datetime.strptime('01/12/2011 16:43:45', '%d/%m/%Y %H:%M:%S')
>>> datetime.datetime(2011, 12, 1, 16, 43, 45)# add proper timezone
pst = ZoneInfo('America/Los_Angeles')
d = d.replace(tzinfo=pst)
>>> datetime.datetime(2011, 12, 1, 16, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
DON’T use datetime.replace to set a timezone with pytz:
import datetime
import pytz### DON'T DO THIS, THIS CODE IS WRONG!!!d = datetime.datetime.utcfromtimestamp(1505325217)
>>> datetime.datetime(2017, 9, 13, 17, 53, 37)pst = pytz.timezone('America/Los_Angeles')
d = d.replace(tzinfo=pst)
>>> datetime.datetime(2017, 9, 13, 17, 53, 37, tzinfo=<DstTzInfo 'America/Los_Angeles' LMT-1 day, 16:07:00 STD>)
Two things are very wrong in the code above:
datetime.replacereplaces thetzinfo: it does not convert the time at all. (utcfromtimestampgives adatetimein UTC, so changingtzinfochanges the time represented)pytzdoesn’t work well withtzinfo. Replacingtzinfowith anything but UTC has some unintended consequences. In the example above, note how the PST timezone setting becomes actually LMT time (Local Mean Time) which is 16:07:00 offset from UTC, NOT 16:00:00.
You end up with a 16h07m offset: bad!
6. datetime to ISO 8601 string
import datetime
from zoneinfo import ZoneInfo
d = datetime.datetime.strptime('01/12/2011 16:43:45', '%d/%m/%Y %H:%M:%S')
>>> datetime.datetime(2011, 12, 1, 16, 43, 45)# add proper timezone
pst = ZoneInfo('America/Los_Angeles')
d = d.replace(tzinfo=pst)
>>> datetime.datetime(2011, 12, 1, 16, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)d.isoformat()
>>> "2011-12-01T16:43:45-08:00"
7. datetime from timestamp
import datetime
from zoneinfo import ZoneInfod = datetime.datetime.utcfromtimestamp(1505325217)
>>> datetime.datetime(2017, 9, 13, 17, 53, 37)
# add timezone info
d = d.replace(tzinfo=ZoneInfo('UTC'))
>>> datetime.datetime(2017, 9, 13, 17, 53, 37, tzinfo=zoneinfo.ZoneInfo(key='UTC'))
8. convert from UTC to another timezone
import datetime
import pytzd = datetime.datetime.utcfromtimestamp(1505325217)
>>> datetime.datetime(2017, 9, 13, 17, 53, 37)
# add timezone info
d = pytz.UTC.localize(d)
>>> datetime.datetime(2017, 9, 13, 17, 53, 37, tzinfo=<UTC>)pst = pytz.timezone('America/Los_Angeles')
d.astimezone(pst)
>>> datetime.datetime(2017, 9, 13, 10, 53, 37, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)
9. Add and Subtract time with timedelta
Daylight saving was on Nov 6th in 2016:
import datetime
import pytzd = datetime.datetime(2016, 11, 5, 16, 43, 45) # naive datetimeutc = pytz.UTC
pst = pytz.timezone('America/Los_Angeles')d = utc.localize(d) # UTC timezone aware
>>> datetime.datetime(2016, 11, 5, 16, 43, 45, tzinfo=<UTC>)# add 1 day to UTC date
d = d + datetime.timedelta(days=1)
>>> datetime.datetime(2016, 11, 6, 16, 43, 45, tzinfo=<UTC>)# now convert to local timezone
d = d.astimezone(pst)
>>> datetime.datetime(2016, 11, 6, 8, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)# daylight saving was respected
DON’T use timedelta with anything but UTC time when using pytz:
import datetime
import pytz### DON'T USE THIS CODE, THIS CODE IS WRONG !!!d = datetime.datetime(2016, 11, 5, 16, 43, 45) # naive datetimeutc = pytz.UTC
pst = pytz.timezone('America/Los_Angeles')d = utc.localize(d) # UTC timezone aware
>>> datetime.datetime(2016, 11, 5, 16, 43, 45, tzinfo=<UTC>)# convert d to 'America/Los_Angeles' timezone
d = d.astimezone(pst)
>>> datetime.datetime(2016, 11, 5, 9, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)# add 1 day to PDT date: DON'T DO THAT
d = d + datetime.timedelta(days=1)
>>> datetime.datetime(2016, 11, 6, 9, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PDT-1 day, 17:00:00 DST>)# daylight saving was not respected, still PDT time, not PST as it should be
So, now the long version:
1. Parsing date & time strings
The first thing is: there are so many date and time formats, whether you are in North America or Europe, whether you speak in 12H or 24H time or even military time:
- “Wednesday September 21st, 06:30PM”
- “09/21/2016 06:30PM”
- “2016/09/21 18:30”
- “21/09/2016 18:30”
Parsing date & time can be a bit of a pain, but that’s not the most complicated thing:
import datetime
format = '%Y-%m-%d %H:%M:%S'
datestring = '2016-09-20 16:43:45'
d = datetime.datetime.strptime(datestring, format)# Note ISO string format is special. More about that later
What we just obtained is called a naive datetime.
Why naive, you ask? Because it is subjective.
2. Time is relative
If it is 16h43 in London right now, it will only be 16h43 (4:43PM) in San Francisco in another 8 hours from now. The same date string can represent different points in time, at different locations.
With a naive datetime, there is no clear way to know at what actual point-in-time an event occurred, and therefore no clear way to order events.
Comes UTC timestamps
3. UTC Timestamps
Universal Time Coordinated (literal translation of the French TUC (Temps Universal Coordiné)
A UTC timestamp is a number of seconds since epoch, which is January 1st 1970 at 00:00 GMT (Greenwich Mean Time)
epoch is defined at a specific place in the world (the GMT timezone) which makes it possible to represent an actual point-in-time.
Note: you may hear about GMT time, UTC time or even Zulu time. These all represent time at the Greenwich Meridian, that happens to go North to South and crosses around Greenwich, in the U.K.
UTC timestamps are useful to order events (log data) and calculate time difference, in terms of elapsed time.
A naive approach to getting a timestamp in python might go like:
# define epoch, the beginning of times in the UTC timestamp world
epoch = datetime.datetime(1970,1,1,0,0,0)now = datetime.datetime.utcnow()
timestamp = (now - epoch).total_seconds()
# subtracting datetime objects result in a datetime.timedelta object which can be expressed in seconds.
This works because the utcnow method returns a naive datetime object in UTC time by default, but don’t do this with a timezone aware dates not in UTC.
We’ll see in a little bit how converting a given date to UTC timestamp, takes a little more effort.
While UTC timestamps are a great way to standardize representation of a point in time, the same UTC timestamp converts to different date strings in different places of the world. It is, after all, defined at GMT.
Using a timestamp, we’ve lost the notion of location, and therefore local time.
So, is it one or the other?
4. Time Offsets, ISO-8601 and Local Time
If we want to know about time-of-day of an event sent by a user, or simply need to represent event times in local time as is the case in most user-facing application, you will need to convert from server time (which should really always be UTC) to local time.
A time offset is the timezone offset that needs to be added to UTC time to represent local time.
ISO-8601 is a standard to represent date strings, including time offsets.
It’s really able to represent date string in many different format, but the most commonly used format in computing is:
2017–09–13T20:58:41+07:00
YYYY-MM-DDTHH:mm:ss+/-HH:mm
In python parsing format that becomes
'%Y-%m-%dT%H:%M:%S%z'to parse an ISO-8601 string in python >3.2, you can use the same function as shown in 1. but if you’re using python 2.7, you’ll need dateutil.parser:
import datetime
import dateutil.parserformat = '%Y-%m-%dT%H:%M:%S%z'
datestring = '2016-09-20T16:43:45-07:00'
d = dateutil.parser.parse(datestring) # python 2.7
d = datetime.datetime.strptime(datestring, format) #3.2+
Now, a very important thing to understand:
An ISO-8601 date string provides:
- local time (the
YYYY-MM-DDTHH:mm:sspart) - a mean to convert to UTC (the time offset, i.e the
+/-HH:mmpart),
However the time offset is only valid for that local date/time.
Yes, time offsets are not set in stone: they actually change, because of daylight saving.
2017–07–13T20:58:41+07:00 or 2017–12–13T20:58:41+08:00 is the time for the same place in the Pacific time zone, but one is in the summer (when Daylight Saving Time DST is observed) and the other is in the winter when it is not.
What that means is that you cannot use just the time offset to assume the location, or compute time deltas.
That will not work well, but you probably won’t notice until there is a DST change, at which point you will probably pull your hair out trying to figure what went wrong.
ISO-8601 string can safely be used to translate local time between time zones, but should not be used as-is to do time manipulations.
5. Timezones
Time offset is not enough. We need timezone information.
What is a timezone?
It’s a little confusing because I’ve talked about UTC time, but UTC is not a timezone. You’ve probably heard about EST, CST, MST or PST if you live in North America, and GMT and CET if you’re in western Europe, and been told these are time zones, but these are not actually timezones, they are naming representing the time offset.
A timezone is usually defined by a continent and a location, like America/Los_Angeles or Europe/Paris or Africa/Tunis
Note there is no notion of Daylight Saving in the name
PST: Pacific Standard Time (GMT-08:00), andPDT: Pacific Daylight Time (GMT-07:00), are both time offset applied in theAmerica/Los_Angelestimezone depending on the time of the year.CET: Central European Time (GMT+01:00)andCEST: Central European Summer Time (GMT+02:00)are both time offsets applied in theEurope/ParistimezoneCET: Central European Time (GMT+01:00)is also applied in theAfrica/Tunistimezone, but daylight saving is not observed, soCETtime is used year round.
to deal with timezone, pytz is your friend… most of the time.
We haven’t mentioned it yet, but when we parsed the ISO-8601 earlier, you may have noticed that the datetime object comes out like:
datetime.datetime(2016, 9, 20, 16, 43, 45, tzinfo=tzoffset(None, -25200))There is a time offset, but not timezone information.
All we can really do with this, is convert back to UTC
import datetime
import dateutil.parser
from zoneinfo import ZoneInfoformat = '%Y-%m-%dT%H:%M:%S%z'
datestring = '2016-09-20T16:43:45-07:00'
d = dateutil.parser.parse(datestring)d = d.replace(tzinfo=ZoneInfo('UTC')) - d.utcoffset()
>>> datetime.datetime(2016, 9, 20, 23, 43, 45, tzinfo=<UTC>)
# now a timezone aware UTC datetime.
Wait, we lost the timezone again.
No we didn’t. We never had the timezone to begin with. We had an offset. As seen above, a timezone may have multiple offsets, so there is no simple way to get a timezone name from the offset.
Timezone name is something you can easily get on the client (browser, in Javascript) and that you should send to your server along with the ISO-8601 string if you need it on the server.
If you know the timezone you want to convert to, then you can do:
import datetime
from zoneinfo import ZoneInfo
import dateutil.parserformat = '%Y-%m-%dT%H:%M:%S%z'
datestring = '2016-09-20T16:43:45-07:00'
d = dateutil.parser.parse(datestring)# UTC datetime
d = d.replace(tzinfo=ZoneInfo('UTC')) - d.utcoffset()pst = ZoneInfo('America/Los_Angeles')
d.astimezone(pst)
>>> datetime.datetime(2016, 9, 20, 16, 43, 45, tzinfo=zoneinfo.ZoneInfo('America/Los_Angeles'))
but don’t get too excited:
datetime operations in localized time can be tricky. Most of the time you want to use UTC, then convert timezone.
Look at this example, where we are a day before Daylight Saving changes, and we want to add one day to the datetime object:
import datetime
from zoneinfo import ZoneInfod = datetime.datetime(2016, 11, 5, 16, 43, 45) # naive datetimeutc = ZoneInfo('UTC')
pst = ZoneInfo('America/Los_Angeles')d = d.replace(tzinfo=utc) # UTC timezone aware
>>> datetime.datetime(2016, 11, 5, 16, 43, 45, tzinfo=zoneinfo.ZoneInfo(key='UTC'))# convert d to 'America/Los_Angeles' timezone
d = d.astimezone(pst)
>>> datetime.datetime(2016, 11, 5, 9, 43, 45, tzinfo=zoneinfo.ZoneInfo(key='America/Los_Angeles'))
d.isoformat()
>>> '2016-11-06T16:43:45-07:00'# add 1 day to PDT date: DON'T DO THAT
d = d + datetime.timedelta(days=1)
>>> datetime.datetime(2016, 11, 6, 9, 43, 45, tzinfo=zoneinfo.ZoneInfo(key='America/Los_Angeles'))d.isoformat()
>>> '2016-11-06T16:43:45-08:00'
# daylight saving was respected, and we added a full day, but that was actually 25h
When using the UTC datetime, and only converting to local time in the end, we have a different effect:
import datetime
from zoneinfo import ZoneInfod = datetime.datetime(2016, 11, 5, 16, 43, 45) # naive datetimeutc = ZoneInfo('UTC')
pst = ZoneInfo('America/Los_Angeles')d = d.replace(tzinfo=utc) # UTC timezone aware
>>> datetime.datetime(2016, 11, 5, 16, 43, 45, tzinfo=<UTC>)# add 1 day to UTC date
d = d + datetime.timedelta(days=1)
>>> datetime.datetime(2016, 11, 6, 16, 43, 45, tzinfo=<UTC>)
d.isoformat()
>>> '2016-11-05T16:43:45+00:00'# now convert to local timezone
d = d.astimezone(pst)
>>> datetime.datetime(2016, 11, 6, 8, 43, 45, tzinfo=<DstTzInfo 'America/Los_Angeles' PST-1 day, 16:00:00 STD>)
d.isoformat()
>>> '2016-11-06T08:43:45-08:00'# daylight saving was respected, but note the hour is now 8, not 9. So here we added a full day (24h) while when working in localized time we added 25h over the DST event.
in short, the choice of the order of operations needs to be carefully considered.
7. Converting a naive date to a timestamp
Now, sometimes you need to deal with naive dates, which means you need to know the timezone in order to do anything useful with them, like convert to a timestamp.
Converting a date to a UTC timestamp, as mentioned above, take a little more effort to make sure the date is in UTC to begin with:
# naive datetime
d = datetime.datetime.strptime('01/12/2011', '%d/%m/%Y')
>>> datetime.datetime(2011, 12, 1, 0, 0)# timestamp will convert the localized naive datetime into a unix timestamp
d.timestamp()
>>> 1322726400.0
Here we go.
I hope this has been useful.
If you only need to remember a few things, this is what you need to remember:
- Always use timezone aware
datetimein UTC on the server, and for any operation, unless you understand the consequences of using localized time. - Use ISO-8601 date strings with offset to transmit date information between client and server.
- Ship the timezone name from the client, don’t try to guess.
Good luck!
